A Conceptual Analysis of Standard Setting in Large - Scale Assessments

Elements of arbitrariness in the standard setting process are explored, and an alternative to the use of cut scores is presented. The first part of the paper analyzes the use of cut scores in large-scale assessments, discussing three different functions: (1) cut scores define the qualifications used in assessments; (2) they simplify the reporting of achievement distributions; and (3) they allow for the setting of targets for such distributions. The second part of the paper gives a decision-theoretic alternative to the use of cut scores and shows how each of the three functions identified in the first part can be approached in a way that may reduce some of the arbitrary nature of standard setting processes. The third part of the paper formulates criteria for standard setting methods that can be used to evaluate their results. (Contains six figures and eight references.) (Author/SLD) ********************************************************************** Reproductions supplied by EDRS are the best that can be made * from the original document. *********************************************************************** A Conceptual Analysis of 00 (-.9) Standard Setting in Research Large-Scale Assessments Report 94-3 U.S. DEPARTMENT OF EDUCATION Once ot Educahonat Research nd Improvement EDUCMIONAL RESOURCES INFORMATION CENTER (ERIC) dsu document has been reoroduceo as rec.ned Nom the parson or orpernzabon OrtgonePng C Mmot changes have been made to onprOne reetOdoCOOn quality Potnts 01 v41.4 0 prontonS stated .n th,s ooc ment do not necessaNy ,ebresent otfictal OEM oose.on or pohcy PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC). Wim J. van der Linden